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FOREWORD 


The  Life  Cycle  System  Management  Model  (LCSMM)  outlines  an  orderly 
series  of  steps  that  must  be  accomplished  by  all  sponsor  and  contractor 
organizations  involved  with  the  development  of  new  Army  materiel  sys¬ 
tems.  TRADOC  Regulation  600-4  outlines  the  Integrated  Personnel  Support 
(IPS)  process  for  Integrating  personnel  and  training  considerations  into 
the  development  effort  for  new  materiel  systems.  This  report  identifies 
and  describes  the  major  missing  information  sources  and  procedures  that 
are  needed  for  accomplishing  the  purposes  of  the  IPS  process.  The  data 
for  this  report  were  obtained  from  an  extensive  review  of  civilian  and 
military  publications  and  from  structured  interviews  with  TRADOC  systems 
managers  (TSMs) ,  their  staffs,  and  related  personnel. 

This  research  is  part  of  ARI's  6.3  program  and  was  accomplished 
under  Army  Project  2Q763743A771 .  It  is  responsive  to  HRN  78-72  entitled 
"Simulation  and  Training  Equipment  Planning  Sources  (STEPS)  Requirements 
for  Weapon  System  Development."  This  HRN  was  submitted  by  COL  John  W. 
Swaren,  Jr.,  Deputy  Commander,  Developing  Systems  Training  and  Devices 
Directorate,  U.S.  Army  Training  Support  Center  (TRADOC),  Fort  Eustls, 
Virginia.  The  advice  and  support  of  COL  Swaren  and  his  staff  were  in¬ 
strumental  to  the  successful  completion  of  this  research.  The  efforts 
of  Ms.  Barbara  Mroczkowski,  who  served  as  COL  Swaren' s  point  of  contact, 
were  especially  helpful  in  completing  thiB  project. 

This  report  will  be  of  particular  interest  to  all  those  who  are 
planning  or  conducting  research  and/or  development  efforts  related  to 
the  LCSMM  or  IPS  process. 


3EPH  ZEIDNER 
£chnical  Director 
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Integrated  Personnel  and  Training  Information 

for  TRADOC  System  Managers  (TSM):  Technological  Gaps 

BRIEF 


Purpose: 

The  Life  Cycle  System  Management  Model  (LCSMM)  outlines  an 
orderly  series  of  steps  that  must  be  accomplished  by  all  sponsor  and 
contractor  organizations  involved  with  the  development  of  new  materiel 
systems.  The  Integrated  Personnel  Support  (IPS)  model  outlines  the 
procedures  that  should  be  followed  during  the  development  of  personnel 
and  training  subsystems  to  comply  with  the  LCSMM.  This  report  iden¬ 
tifies  some  of  the  technological  problems  that  must  be  solved  to  faci¬ 
litate  the  application  of  the  IPS  model. 


Procedure: 

The  technological  problems  or  gaps  reported  on  were  identified 
during  an  extensive  literature  review  for,  and  subsequent  development 
of,  a  handbook  for  TRADOC  System  Managers  (TSM),  and  through  inter¬ 
views  with  TSMs  and  their  staffs.  The  handbook  describes  information 
sources  and  technical  procedures  related  to  each  of  the  IPS  model 
events  discussed  in  TRADOC  Regulation  600-4,  Integrated  Personnel 
Support.  Where  possible  technical  sources  of  information  that 
described  how  to  accomplish  each  IPS  event  are  included.  When  satis¬ 
factory  documentation  could  not  be  found,  a  technological  gap  was  said 
to  exist.  In  addition  to  technical  reports,  texts  and  handbooks,  an 
extensive  search  was  made  of  military  publications,  especially  ISO 
manuals.  Army  regulations,  training  circulars  and  pamphlets,  and 
military  specifications  and  standards.  Also  as  part  of  this  study 
TRADOC  System  Managers  (TSM)  and  their  staffs,  and  other  persons 
tasked  by  TSMs,  were  interviewed  for  the  purpose  of  identifying  their 
information  requirements. 


Findings: 

Adequate  techniques  do  not  exist  for  identifying  training  and 
personnel  requirements  during  the  early  stages  of  the  Life  Cycle 
Management  Model.  During  the  Conceptual  Phase  of  the  LCSMM  in  par¬ 
ticular,  there  is  a  lack  of  techniques  for  generating  data  needed  to 
make  estimates  of  training  and  personnel  requirements.  Moreover, 
there  is  a  lack  of  techniques  for  predicting  the  impact  of  materiel 
design  on  training/ personnel  subsystems,  making  it  difficult  to 
develop  defensible  training  plans  and  cost-effectiveness  estimates 
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during  the  Concept  Phase  of  the  LCSMM.  More  specifically,  there  is 
an  absence  of  satisfactory  techniques  for: 

•  deducing  training  and  personnel  requirements  from 
materiel  characteristics,  or  the  reverse 

•  identifying  excessive  human  resources  demands  stemming 
frorti  materiel  concepts 

•  identifying  functions  most  appropriately  performed  by 
equipment,  persons,  or  by  a  man-machine  combination 

•  generating  task  analytic  data  during  Phase  I  of  the 
LCSMM 

•  identifying  high-risk  training  tasks  during  Phase  I 
of  the  LCSMM. 

•  indicating  appropriate  training  strategies  before 
actual  hardware  is  developed 

•  developing  estimates  of  training  costs  and  training 
effectiveness  for  alternative  training  strategies 

•  describing  human  resources  data/requirements/constraints 
in  terms  that  are  meaningful  to  design  engineers 

It  was  concluded  that  solutions  to  the  foregoing  technological 
problems  must  be  developed.  If  not,  it  seems  unlikely  that  the  gains 
expected  through  application  of  the  LCSMM  and  IPS  models  will  be 
achieved. 


Utilization  of  Findings: 

The  results  of  this  technical  report  will  be  used  primarily  as  a 
planning  document  for  further  research  designed  to  enhance  the  early 
development  of  the  personnel  and  training  subsystems  of  developing 
weapons  systems.  This  document  will  be  of  interest  to  all  those  who 
are  planning  research  and  development  efforts  that  are  related  to 
either  the  IPS  or  LCSMM  models. 
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INTEGRATED  PERSONNEL  AND  TRAINING  INFORMATION 
FOR  TRADOC  SYSTEM  MANAGERS  (TSM) : 
TECHNOLOGICAL  GAPS 


STUDY  OBJECTIVES 

This  report  reviews  selected  technological  gaps  and  problem  areas 
in  existing  procedures  for  the  development  of  training  and  personnel 
system  development  procedures.  The  report  concentrates  on  deficien¬ 
cies  noted  in  the  application  of  such  procedures  during  the  develop¬ 
ment,  earlier  in  this  study,  of  a  handbook  on  training  management 
guidelines  for  use  with  new  materiel  systems. 


BACKGROUND 

During  recent  years  the  development  of  improved  procedures  for 
designing  personnel  and  training  systems  has  received  considerable 
attention.  Numerous  documents  containing  guidance  material  for  the 
design  and  implementation  of  training  and  personnel  support  systems 
have  been  prepared.  Recognizing  the  management  and  coordination 
problems  involved  with  such  efforts,  the  Army  has  established  TRADOC 
System  Manager  (TSM)  offices  in  the  U.  S.  Army  Training  and  Doctrine 
Command.  TSMs  "are  responsible  for  the  integrated  personnel  support 
(IPS)  management  of  selected  systems  within  TRADOC  to  ensure  that  per¬ 
sonnel  and  training  requirements  are  developed  and  fully  integrated 
early  and  continuously  throughout  the  development  cycle"  (TRADOC 
Regulation  600-4).  To  aid  the  TSM,  handbooks  have  been  developed  that 
describe  in  general  terms  the  activities  and  products  for  which  the 
TSM  is  responsible  (Hanson  and  Purifoy,  1978;  MITRE  Corp. ,  1977). 

As  part  of  the  present  study,  a  handbook  for  TSMs  (Kinton,  1979) 
was  prepared  to  provide  "a  basic  reference  source  of  training  manage¬ 
ment  guidelines  and  procedures  for  use  in  the  life  cycle  development 
of  Army  materiel  systems."  For  each  event  described  in  TRADOC 
Regulation  600-4,  Integrated  Personnel  Support,  this  handbook  outlines 
the  general  procedures  for  accomplishing  the  event,  the  sources  of 
everjt  input  data,  and  the  use  of  event  products  and  output  data. 

During  the  development  of  this  handbook,  certain  technological 
gaps  were  identified.  The  major  gaps  so  identified  are:  the  lack 
of  needed  data  at  various  points  in  the  formal  system  development 
process;  the  lack  of  techniques  for  predicting  the  impact  of  materiel 
design  considerations  on  training/personnel  subsystems,  and  the 
reverse;  and  the  difficulty  of  developing  defensible  training  plans 
and  cost  estimates  during  the  early  stages  of  materiel  development. 
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The  development  of  solutions  to  these  and  other  technical  problems 
would  greatly  enhance  the  capability  of  training  and  personnel  system 
developers. 

The  design  of  new  materiel  systems  involves  both  the  design  of 
hardware  and  the  design  of  an  integrated  support  system,  elements  of 
which  involve  personnel  and  training  (AR  700-127).  The  orderly 
and  effective  development  of  a  new  system  must  include  the  concurrent 
development  of  appropriate  training  and  personnel  subsystems. 

Training  and  personnel  support  subsystems  may  be  more  expensive  than 
their  materiel  counterparts.  Because  of  this  cost  factor  and  antici¬ 
pated  shortfall  of  personnel  for  manning  future  weapons  systems,  it  is 
imperative  that  future  systems  be  designed  to  minimize  the  require¬ 
ments  for  personnel  and  the  costs  for  training. 

Increased  interest  in  the  early  design  of  personnel  and  training 
subsystems  has  heightened  our  awareness  that  our  ability  to  identify 
the  relationship  between  equipment  design  and  personnel  and  training 
requirements  is  both  underdeveloped  and  underutilized.  Currently 
training  system  development  is  based  on  the  application  of  instruc¬ 
tional  system  development  (ISD)  procedures  (TRADOC  Pamphlet  350-30). 
These  procedures  describe  how  to  generate  and  process  data  during  a 
task  and  skill  (front-end)  analysis,  a  type  of  analysis  that  depends 
on  the  existence  of  hardware.  This  conventional  task  and  skill  analy¬ 
sis  is  difficult  to  perform  on  an  equipment  concept,  and  often  is  not 
conducted  until  a  prototype  of  the  new  equipment  becomes  available. 

By  the  time  such  prototypes  are  available  for  analysis,  it  is  usually 
too  late  for  personnel  and  training  considerations  to  have  an  impact 
on  equipment  design. 

Also,  within  the  realm  of  training,  current  development  proce¬ 
dures  do  not  enable  us  to  make  reliable  comparisons  between  alter¬ 
native  training  options,  especially  when  the  comparisons  involve  a 
determination  of  training  and  personnel  costs  over  the  lifetime  of  a 
proposed  system. 

Historically,  the  development  of  new  systems  involved  the  design 
of  hardware  to  meet  a  set  of  mission/functional  requirements.  Often 
it  was  tacitly  assumed  that  the  human  components  of  the  new  system 
could  compensate  for  deficiencies  in  system  hardware  or  for  functional 
requirements  that  could  not  be  met  by  the  hardware  portion  of  tlv 
system.  The  approach  was  to  design  the  materiel  in  keeping  with 
known  physical  and  physiological  limitations  of  personnel  and  then  to 
hope  that  through  selection  and  training  a  cost-effective  man-machine 
system  could  be  produced. 

This  approach  often  resulted  in  the  development  of  man-machine 
systems  that  function  less  than  optimally  in  the  field  because 
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excessive  requirements  are  imposed  on  operator  and/or  or  maintenance 
personnel.  Such  problems  tended  to  be  compounded  by  the  belief  that 
the  development  of  a  training  subsystem  could  not  begin  until  design 
of  the  materiel  had  been  completed.  Consequently,  it  was  not  unusual 
for  new  hardware  to  be  fielded  in  the  absence  of  suitable  numbers  and 
types  of  persons  to  man  the  system. 

In  recognition  of  the  foregoing  problems,  the  Army  developed  the 
Life  Cycle  System  Management  Model  (LCSMM)  for  new  systems.  This  model 
outlines  an  orderly  series  of  steps  that  must  be  accomplished  by  all 
sponsor  and  contractor  organizations  involved  with  the  development  of 
a  new  system  (DA  Pamphlet  11-25).  The  goal  of  the  LCSMM  is  to  pro¬ 
mote  the  logical,  coordinated,  and  carefully  reasoned  development  of  a 
total  system. 

As  shown  in  Figure  1  (17),  the  LCSMM  subdivides  the  total  life 
cycle  of  materiel  systems  into  four  phases— the  concept  phase,  the 
validation  phase,  the  full-scale  engineering  development  phase,  and 
the  production  and  deployment  phase.  Numerous  regulations,  pamphlets, 
circulars,  and  handbooks  have  been  prepared  to  describe  the  activities 
that  should  occur  during  each  of  these  four  phases,  including  assigned 
responsibilities  and  the  recommended  phasing  for  various  activities. 
The  LCSMM  stresses  the  coordinated  development  of  all  logistic  support 
systems  in  concert  with  new  materiel.  Milestones  or  checkpoints  have 
been  established  so  that  independent  reviewers  can  certify  that  all 
activities  for  any  particular  phase  have  been  accomplished  and  docu¬ 
mented  satisfactorily  before  the  system  is  allowed  to  progress  to  the 
next  phase  of  development. 

During  Phases  I  and  II  of  the  LCSMM,  all  portions  of  the  new 
system  progress  from  a  general  concept  stage  through  the  development 
and  test  of  prototype  materiel  and  procedures.  During  this  process 
many  design  options  may  be  considered,  both  for  materiel  and  for  per¬ 
sonnel  and  training  subsystems.  To  aid  in  the  selection  of  the  most 
appropriate  option,  numerous  formal  and  informal  trade-off  studies  may 
be  performed.  Some  of  these  studies  may  examine  alternative  personnel 
and  training  approaches  in  an  effort  to  identify  those  that  are  most 
cost-effective. 

Of  particular  interest  to  this  report  is  the  identification  of 
the  points  in  Phases  I  and  II  of  the  LCSMM  at  which  human  resources 
data  and  techniques  can  have  an  impact  on  materiel  design.  The  ulti¬ 
mate  goal  of  the  materiel  development  process  is  to  design  a  total 
system  providing  a  cost-effective  way  to  accomplish  the  mission  needs 
that  initiated  the  development  effort.  The  final  system  configuration 
should  be  a  cost-effective  mix  of  materiel,  personnel,  training,  and 
other  elements  of  the  logistics  support  package. 
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Figure  1.  Systems  Acquisition  Cycle 
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METHODOLOGY 


This  report  discusses  the  needs  for  improved  procedures  for  deve¬ 
loping  and  making  use  of  human  resources  data  during  the  design  of  new 
materiel  systems.  These  needs,  or  technological  gaps,  were  identified 
during  the  extensive  literature  review  for,  and  subsequent  development 
of,  the  handbook  for  TRADOC  system  managers  (Kinton,  1979)  and  through 
interviews  with  TSMs  and  their  staffs. 

For  this  report  a  "technological  gap"  is  defined  as  the  absence 
of  documented  procedural  guidance  describing  how  to  accomplish  a  major 
event  during  the  development  of  training  or  personnel  subsystems. 

This  definition  allows  the  inclusion  of  a  wide  range  of  problem  areas. 

A  study  of  any  set  of  procedures--media  selection  procedures,  for 
example--will  show  that  they  vary  considerably  in  terms  of  specificity. 
At  one  extreme  there  are  general  media  selection  strategies  that  yield 
general  procedural  guidelines  which  can  be  implemented  by  experienced 
personnel.  At  the  other  extreme  there  should  be  specific  "fully  pro¬ 
cedural  ized  media  selection  job  performance  aids,"  which  provide 
detailed,  step-by-step  instructions  and  are  designed  for  use  by  in¬ 
experienced  personnel.  In  actuality,  such  aids  do  not  exist. 

Military  specification  MIL-T-29053  (1977)  is  probably  the  closest 
approximation  to  such  a  fully  procedural ized  aid. 

Experience  gained  during  this  research  indicated  that  guidance 
material  related  to  training  and  personnel  subsystem  development 
usually  falls  between  the  above  two  extremes.  Many  textbooks,  tech¬ 
nical  reports,  taxonomies,  etc.  provide  the  informational  content 
one  needs  to  develop  one's  own  set  of  detailed  procedures.  Documents 
prepared  by  the  military— ISD  manuals  (Berkowitz  and  O'Neal,  In  press), 
training  circulars  and  pamphlets— military  specifications  and  stand¬ 
ards— attempt  to  provide  more  detailed  procedural  guidance.  Even  in 
these  documents,  however,  procedures  for  making  key  decisions  may  be 
omitted,  implying  that  they  should  be  made  on  the  basis  of  experience 
plus  an  understanding  of  the  objectives  that  must  be  accomplished. 

As  an  illustration,  TRADOC  Pamphlet  351-4  contains  a  detailed 
discussion  of  the  procedures  for  selecting  tasks  for  training.  How¬ 
ever,  these  procedures  are  based  on  the  use  of  either  four  or  eight 
factors  that  have  to  be  weighted.  They  either  receive  equal  weight 
(by  default)  or  are  assigned  weights  based  on  a  command  decision, 
which  in  turn  usually  is  based  on  the  professional  judgment  of  exper¬ 
ienced  combat  and  training  developers. 

Technological  gaps  as  defined  and  discussed  in  this  report  will 
not  be  eliminated  by  the  preparation  of  descriptions  of  strategy  and 
general  procedures.  They  would  be  eliminated  by  the  preparation  of 


5 


f 


fully  procedural i zed  performance  aids.  However,  this  latter  approach 
would  be  an  extremely  lengthy  and  expensive  undertaking. 

A  more  suitable  approach  to  eliminating  these  technological  gaps 
might  be  to  analyze  training  development  events  into  their  respective 
tasks,  prepare  procedural  guidance  covering  these  tasks,  and  prepare 
training  material  to  cover  the  accomplishment  of  elements  of  those 
tasks.  Essentially,  this  is  the  task  analytic  approach  that  the  Army 
now  follows  when  developing  Skill  Performance  Aids  (SPA). 


TECHNOLOGICAL  GAPS— MATERIEL  DESIGN 


DEDUCING  TRAINING  AND  PERSONNEL  REQUIREMENTS 
FROM  MATERIEL  CHARACTERISTICS 

The  successful  development  of  major  materiel  systems  is  the 
culmination  of  a  complex  series  of  compromises  and  trade-offs.  The 
desire  for  certain  equipment  capabilities  sometimes  must  be  discarded 
because  these  capabilities  are  not  feasible  within  the  present  state 
of  the  art.  The  desire  to  meet  certain  materiel  or  operational 
requirements  must  sometimes  give  way  when  it  is  confronted  with  the 
realization  that  the  people  needed  to  man  the  proposed  system  cannot 
be  provided.  In  recognition  of  these  realities,  the  LCSMM  was 
designed  so  that  materiel-personnel  trade-offs  can  be  evaluated  at 
least  twice  (OT  I  and  II)  before  the  materiel  enters  into  production. 

Until  recently  it  was  almost  inevitable  that  when  materiel  and 
personnel  considerations  clashed,  materiel  requirements  prevailed.  In 
part  this  situation  stemmed  from  the  common  assumption  that  the  deve¬ 
lopment  of  training  programs  must  await  the  design  of,  and  be  subser¬ 
vient  to,  materiel  developments.  Practically  all  the  concepts  and 
procedural  guidelines  that  have  been  formulated  on  how  to  design 
training  subsystems  are  based  on  the  assumption  that  training  require¬ 
ments  will  be  deduced  from  materiel  characteristics.  Thus,  during  the 
development  of  training  and  personnel  supply  subsystems  for  new 
materiel,  almost  all  activities  are  directed  toward  (a)  defining  and 
refining  qualitative  and  quantitative  personnel  requirements;  and  (b) 
designing  and  refining  individual  and  collective  training  plans. 

During  Phase  I,  the  Concept  Phase,  of  the  LCSMM,  materiel 
requirements  are  specified  and  tentative  personnel  and  training 
requirements  are  identified.  A  major  technological  gap  that  exists 
relative  to  these  activities  concerns  our  lack  of  knowledge  about  the 
relationship  between  equipment  characteristics  and  personnel  require¬ 
ments.  As  suggested  by  Meister  (1973), 
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There  is  a  need  for  improved  guidance  for  deducing 
personnel  numbers  and  type  requirements  from  materiel 
characteristics.  In  particular  there  is  a  need  for 
data  relating  equipment  dimensions  to  skill  dimensions. 

During  the  initial  design  of  materiel,  numerous  configurations 
may  be  considered.  Alternatives  usually  are  discarded  for  other  than 
human  resources  reasons  until  one  or  a  few  options  remain.  Within 
each  of  these  options  alternative  subsystem  designs  options  may  have 
been  identified.  For  each  of  these  subsystem  alternatives,  two 
related  questions  need  to  be  answered:  (a)  Given  a  set  of  materiel 
alternatives,  do  the  alternatives  have  different  personnel  or  training 
implications,  and  (b)  given  any  specific  alternative,  how  many  persons 
by  MOS  and  skill  level  are  required  to  support  the  system?  Data 
relating  equipment  dimensions  to  skill  dimensions  would  be  of  con¬ 
siderable  aid  when  developing  answers  to  such  questions. 

Comparability  Analysis 

A  common  way  of  estimating  personnel  and  training  requirements 
for  new  materiel  involves  the  use  of  comparability  analysis  procedures 
(Goclowski,  King,  Ronco  and  Askrew,  1978).  The  concept  of  com¬ 
parability  analysis  is  widely  known  and  practiced.  It  involves  com¬ 
paring  features  of  proposed  materiel  with  features  of  similar  extant 
equipment.  If  comparable  subsystems  and  components  are  identified, 
personnel  and  training  data  related  to  those  subsystems/components  are 
used  to  estimate  human  resources  requirements.  When  comparable  sub¬ 
systems  and  components  cannot  be  identified,  estimates  of  human 
resources  requirements  are  based  on  expert  opinion.  In  practice,  per¬ 
sonnel  and  training  estimates  for  new  materiel  made  during  the  concept 
phase  for  that  equipment  usually  are  based  on  a  mix  of  historical  data 
and  expert  opinion. 

The  U.S.  Air  Force  has  been  actively  investigating  and  refining 
comparability  analysis  techniques,  yet  considerable  Improvements  to 
the  techniques  are  still  needed.  In  particular,  Goclowski  et  al. 
(1978s,  1978b)  report  that  the  techniques  do  not  seem  to  be  exten¬ 
sively  employed  during  the  Concept  Phase,  although  this  is  probably 
the  most  profitable  locus  of  application.  Therefore,  as  a  tech¬ 
nological  gap: 

There  is  a  need  for  improved  guidance  for 
identifying  comparable  systems  and  for- 
historical  data  to  help  identify  personnel 
and  training  requirements,  especially  during 
Phase  I  of  the  LCSMM. 
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Identification  of  Excessive  Human  Resources  Demands 


The  field  of  human  engineering  came  about  partly  because  of  the 
need  to  know  how  to  design  materiel  that  could  be  operated  by  persons 
of  specified  physical  and  mental  attributes.  It  is  now  possible  to 
describe  quite  accurately  how  operator  stations  should  be  designed, 
what  environmental  conditions  should  not  be  exceeded,  and  so  on. 
However,  it  still  is  difficult  to  identify,  on  the  basis  of  materiel 
and/or  operational  concepts,  when  personnel  requirements  are 
excessive.  We  still  design  equipment  that  overloads  the  operator  with 
data,  and  we  still  propose  operational  concepts  that  probably  will 
result  in  excessive  physical  and  mental  fatigue  before  a  typical 
mission  is  completed.  Both  Meister  (1973)  and  Goclowski  et  al .  (1978) 
discuss  some  of  the  problems  in  this  area  and  as  a  technological 
gap  conclude  that: 

There  is  a  need  for  techniques  for  reviewing 
materiel  and  the  proposed  operational  use  of 
materiel  in  order  to  identify  where  excessive 
demands  may  be  placed  on  human  resources. 

Allocations  of  Functions  to  Equipment  and  Personnel 

In  a  text  by  Meister  (1973),  excessive  personnel  demands  are 
discussed  in  the  context  of  function  allocation.  During  the  initial 
design  of  a  new  system,  assumptions  are  made  about  which  functions 
will  be  performed  by  equipment  and  which  by  individuals  or  crews. 
Engineering  handbooks  provide  general  guidance  for  identifying  those 
types  of  functions  best  performed  by  humans.  Meister  believes  that 
this  guidance  needs  considerable  improvement  and  as  a  technological 
gap  reports  that: 

There  is  a  need  for  additional  research  on  the 
conditions  under  which  a  function  or  task  should 
be  assigned  to  operator  personnel  as  opposed  to 
the  materiel. 


DEDUCING  MATERIEL  CHARACTERISTICS  FROM 
HUMAN  RESOURCES  CONSTRAINTS 

The  discussion  so  far  has  been  focused  on  the  typical  materiel 
development  process— materiel  requirements  are  identified  and  then 
personnel  and  training  requirements  are  deduced.  In  the  past  few 
years  it  has  become  obvious  that  this  approach  must  be  modified. 
Currently  and  in  the  future,  people  requirements  may  be  more  difficult 
to  meet  than  materiel  requirements.  The  traditional  design  rela¬ 
tionship  between  materiel  and  personnel  requirements  may  have  to  be 
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reversed,  placing  both  the  materiel  and  the  training/personnel  devel¬ 
opers  into  an  unexplored  area.  In  the  immediate  future  the  general 
requirement  posed  to  the  materiel  developer  may  be:  Given  that  the 
materiel  will  be  manned  by  persons  of  a  specific  MOS  and  skill  level, 
and  that  personnel  quantities  and  training  requirements  will  not 
exceed  specified  limits,  design  the  materiel  to  meet  these 
constraints. 

As  noted  earlier,  better  guidance  is  needed  for  deducing  person¬ 
nel  requirements  from  materiel  characteristics.  A  more  general  state¬ 
ment  of  this  requirement,  or  technological  gap,  is: 

There  is  a  need  for  more  information  on  the 
relationship  between  equipment  design  and 
numbers  and  types  of  personnel  required. 

This  information  should  provide  the  basis  for 
deducing  materiel  requirements  from  personnel 
and  training  requirements/constraints,  and 
vice  versa. 

Currently,  engineers  do  not  know  how  to  design  equipment  to  meet 
specific  skill  constraints  nor  do  they  know  how  to  design  equipment  to 
achieve  a  reduction  in  skill  level  requirements.  Moreover,  equipment 
designers  cannot  answer  such  questions  as:  Given  a  specific  number  of 
persons  as  equipment  operators,  how  should  the  equipment  be  configured 
to  provide  the  most  efficient  performance? 

At  one  time  it  was  believed  that  technological  advances,  espe¬ 
cially  in  automation,  would  result  in  materiel  that  could  be  operated 
effectively  by  less  skilled  personnel.  So  far  the  opposite  has  proven 
to  be  the  case.  A  better  understanding  of  the  relationship  between 
equipment  dimensions  and  personnel  requirements  might  have  forewarned 
us  of  this  outcome.  Still,  it  seems  helpful  to  explore  ways  to  make 
better  use  of  less  skilled  personnel.  Is  it  possible  to  trade  per¬ 
sonnel  numbers  for  skill  level  so  that  more  persons  of  a  lower  skill 
can  operate  equipment  as  well  as  fewer  persons  of  a  higher  skill? 

This  and  similar  questions  are  worthy  of  continued  study. 


INCREASING  THE  IMPACT  OF  HUMAN  RESOURCES  CONSIDERATIONS 
ON  MATERIEL  DESIGN 

At  numerous  points  during  Phases  I  and  II  of  the  LCSMM,  materiel 
design  may  be  affected  by  human  resources  considerations  and 
constraints.  During  Phase  I,  training  and  personnel  requirements  are 
identified  and  described  In  a  Letter  of  Agreement  (LOA).  While  these 
requirements  are  being  developed.  Information  may  be  generated  to 
suggest  that  human  resources  constraints  are  being  exceeded.  During 
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the  allocation  of  functions  to  equipment  and  to  personnel  (Event  A6  of 
the  IPS  model,  TRADOC  Regulation  600-4),  a  further  indication  can  be 
derived  as  to  whether  proposed  operators  can  or  cannot  perform  the 
functions  assigned  to  them.  From  a  Cost  and  Effectiveness  Analysis 
(CEA)  of  proposed  training  plans,  estimates  can  be  made  of  the  extent 
to  which  cost  or  training  time  constraints  may  be  exceeded.  Finally, 
during  preparation  of  the  Advanced  Development  Contract,  the  materiel 
and  training/personnel  developers  again  have  an  opportunity  to  judge 
whether  human  resources  requirements  are  excessive. 

The  manner  in  which  human  resources  data  are  used  by  design  engi¬ 
neers  has  received  considerable  study.  Meister  (1973)  and  Kidd  and 
VanCott  (1972)  note  that  many  crucial  design  decisions  are  made  during 
preparation  of  the  AD  contract  proposal  and  are  never  controverted.  A 
proposal  preparation  team  for  a  large  system  will  include  human  fac¬ 
tors  engineers,  especially  if  the  contract  states  that  HFE  will  be 
considered  when  developing  the  design  for  the  proposed  system. 

However,  the  mere  presence  of  HF  personnel  is  not  enough.  These  per¬ 
sons  must  be  willing  to  offer  specific  suggestions  in  response  to  spe¬ 
cific  questions  about  personnel  and  training  (1972).  Also,  they  must  be 
willing  to  perform  a  rough  task  and  skill  analysis  (TASA)  quickly  and 
on  the  basis  of  materiel  concepts.  This  analysis  should  concentrate 
on  (a)  identifying  excessive  operator  requirements;  (b)  calculating 
results/impact  of  excessive  requirements;  and  (c)  identifying  equip¬ 
ment  characteristics  that  might  minimize  personnel  and  training 
problems  (10).  The  techniques  for  accomplishing  the  TASA  in  the 
absence  of  equipment  have  not  been  developed.  Thus: 

There  is  a  need  for  procedures  describing  how 
to  quickly  perform  task  and  skill  analysis  as 
part  of  the  process  of  refining  design  concepts, 
and  then  translating  the  data  into  estimates  of 
training  and  personnel  requirements. 

Askren  (1973)  and  Chapanis  (1976)  have  noted  that  the  way  person¬ 
nel  and  training  requirements/constraints  are  typically  worded  has 
little  meaning  for  a  design  engineer.  The  statement  that  the  equip¬ 
ment  must  be  designed  to  be  operated  by  persons  holding  MOS  XXX  at  the 
3  skill  level  has  no  meaning  to  an  engineer.  His  only  recourse  is  to 
inquire  whether  similar  extant  equipment  can  be  operated  by  such  per¬ 
sonnel.  If  so,  that  is  the  end  of  his  consideration  of  personnel 
requirements. 

As  another  illustration,  the  requirement  that  the  total  training 
program  for  the  new  system  not  exceed  a  cost  of  $5  million  per  year, 
adjusted  for  inflation,  has  no  design  implication  for  equipment 
designers.  These  illustrations  indicate  that: 
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There  is  a  need  for  guidelines  on  how  personnel 
and  training  recommendations/constraints  should 
be  worded  so  that  design  engineers  can  understand 
them  and  use  them. 

There  is  a  need  for  ways  to  display  human  re¬ 
sources  data  so  that  their  impact  on  materiel  design 
or  on  training  and  personnel  requirements  becomes 
more  obvious. 

The  need  for  improved  ways  to  display  human  resources  data  will  be 
discussed  later  in  this  report. 


TECHNOLOGICAL  GAPS— TRAINING  PROGRAM  DESIGN 

The  state  of  the  art  for  training  technology  has  shown  substantial 
advances  in  the  past  two  decades.  Instructional  system  design  proce¬ 
dures  were  developed,  and  refinements  continue  to  be  made.  Task  and 
skill  analysis  techniques  have  been  considerably  refined,  especially 
as  applied  to  the  preparation  of  SPA  materiel.  Practically  all  these 
advances,  however,  are  based  on  the  generation  of  data  from  existing 
systems.  As  already  noted  in  this  report,  techniques  for  applying 
training  technology  during  the  Concept  Phase  of  new  materiel  develop¬ 
ment  have  not  been  well  developed.  Some  of  the  technological  gaps  in 
this  area  will  now  be  reviewed. 


TASK  AND  SKILL  ANALYSIS  FOR  DEVELOPING  SYSTEMS 

TRADOC  Regulation  351-4  emphasizes  the  importance  of  select¬ 
ing  tasks  for  training  during  the  early  stage  of  the  training  develop¬ 
ment  process.  This  regulation  also  stresses  the  importance  of 
developing  a  complete  inventory  of  tasks  prior  to  task  selection. 

Some  of  the  problems  associated  with  developing  a  task  inventory 
for  new  materiel  already  have  been  discussed.  The  techniques  for 
developing  the  inventory  depend  in  part  on  comparability  analysis 
techniques,  and  these  techniques  need  further  refinement.  Also,  the 
inventory  depends  in  considerable  measure  on  how  functions  are  allo¬ 
cated  between  operator  positions  and  the  equipment.  Function  alloca¬ 
tion  is  another  area  in  which  adequate  techniques  do  not  exist. 

Chapanis  (1976)  has  discussed  the  difficulty  of  anticipating  the 
need  for  training  to  handle  emergencies.  He  notes  that  with  highly 
automatic  systems,  operators  monitor  system  operation  until  something 
happens.  Then  they  must  respond  rapidly  and  accurately.  For  systems 
in  the  development  stage,  it  is  difficult  to  anticipate  emergency 
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situations;  it  is  easy  to  overlook  tasks  that  must  be  performed  to 
correct  operational  emergencies  or  hazardous  situations.  The  dif¬ 
ficulty  of  identifying  situations  in  which  excessive  operator  require¬ 
ments  may  result  from  materiel  design  or  operational  concepts  has 
already  been  noted. 

Improved  procedures  for  anticipating  these  situations  should  be 
developed.  In  all  probability,  emergency  situations  would  be  identi¬ 
fied  as  mission-critical  or  high-risk  tasks  for  which  appropriate 
training  material  should  be  developed.  In  extreme  instances,  person¬ 
nel  studies  during  the  Concept  Phase  might  demonstrate  that  the  pro¬ 
posed  design  of  the  hardware  must  be  modified  because  personnel 
training  and/or  selection  procedures  cannot  be  expected  to  provide 
persons  able  to  cope  with  one  or  more  of  the  emergency  situations. 

The  training  development  procedures  that  can  be  utilized  in  the 
above  problem  areas  are,  for  the  most  part,  applicable  primarily  to 
individual  tasks.  Analysis  techniques  for  developing  an  inventory  of 
collective  tasks  (tasks  peformed  by  crews  or  units)  are  seldom 
described  in  the  literature.  The  most  recent  guidance  for  task  and 
skill  analysis  (TRADOC  Pamphlet  351-4  and  MIL-M-63035)  does  not 
address  the  analysis  of  collective  tasks.  In  view  of  these  con¬ 
siderations  it  appears  that: 

There  is  a  need  for  better  techniques  for  apply¬ 
ing  task  and  skill  analysis  procedures,  especially 
those  related  to  the  development  of  task  inventories, 
during  the  Concept  Phase  of  the  LCSMM.  In  addition, 
there  is  a  need  for  improved  guidance  for  developing 
an  inventory  of  collective  (crew)  tasks  during  all 
stages  of  the  LCSMM. 

Selection  of  Tasks  for  Training 

A  training  system  does  not  attempt  to  provide  training  for  all 
individual  and  collective  tasks.  Many  tasks  are  simple  enough,  or 
common  enough,  that  training  is  not  required.  There  are  tasks  that 
can  be  described  well  enough  in  technical  manuals  or  in  job  perfor¬ 
mance  aids  so  that  training  is  not  required.  Also,  there  are  tasks 
that  must  be  taught  by  informal  on-the-job  training  because  funds  are 
not  available  to  cover  them  during  institutional  training  or  in  exten¬ 
sion  training  material.  Finally,  there  are  some  tasks  that  are  not 
trainable. 

The  selection  of  tasks  for  training  is  an  extremely  important 
activity.  For  developing  systems,  training  task  selection  is  a  designated 
milestone.  It  is  an  activity  performed  jointly  by  the  materiel  and 
training  developers,  and  the  result  is  an  agreed-upon  list  of  tasks 
that  will  be  covered  by  a  training  program  or  by  SPA  material. 
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ISO  manual S--TRADOC  Pamphlet  350-30,  and  especially  TRADOC 
Pamphlet  351-4  devote  considerable  attention  to  the  task  selection 
process.  The  most  time-consuming  and  difficult  task  selection  model 
considers  eight  factors:  percent  of  persons  performing  task,  percent 
of  time  spent  performing  task,  consequences  of  inadequate  performance, 
task  delay  tolerance,  frequency  of  performance,  task  learning  dif¬ 
ficulty,  probability  of  deficient  performance,  and  immediacy  of  per¬ 
formance  or  time  between  job  entry  and  task  performance.  A  simpler 
version  of  this  model,  bases  task  selection  on  only  four  factors:  per¬ 
cent  of  persons  performing  task,  task  delay  tolerance,  consequences  of 
inadequate  performance,  and  task  learning  difficulty.  These  are  the 
four  factors  that  many  practitioners  judge  to  be  of  greatest  impor¬ 
tance.  Both  of  these  models  employ  data  that  may  be  difficult  to 
generate  for  developing  systems,  especially  during  the  Concept  Phase 
when  the  model  must  first  be  applied. 

A  third  model,  the  DIF  Model,  selects  tasks  on  the  basis  of 
supervisor,  job  incumbent,  and/or  subject  matter  experts'  judgments 
about  difficulty  (D),  importance  (I),  and  frequency  (F). 

More  research  is  required  on  the  ease  of  use  and  the  validity 
of  each  of  these  models.  In  particular,  further  guidance  is  needed 
on  how  the  factors  in  each  of  these  models  should  be  weighted. 
Currently  weighting  seems  to  be  based  on  a  command  position  that  is 
developed  on  the  basis  of  professional  judgment. 

To  further  emphasize  the  importance  of  task  selection  during 
Phase  I  of  the  LCSMM,  it  should  be  noted  that  data  based  on  this  acti¬ 
vity  are  used  to  prepare  an  Outline  Individual  and  Collective  Training 
Plan  (OICTP),  which  in  turn  is  used  to  prepare  a  Training  Support 
Plan.  The  result  is  a  refined  OICTP  that  provides  the  data  for  a  Cost 
Effective  Analysis  of  the  proposed  training  plans— the  analysis 
that  provides  the  basis  for  judging  whether  the  training  plans  are 
acceptable.  If  they  are  accepted,  the  OICTP  identifies  those  tasks 
for  which  training  material /programs  will  be  prepared  prior  to  0T  I. 
Thus,  the  selection  of  tasks  for  training  during  Phase  I  merits 
increased  emphasis.  In  view  of  these  considerations  it  seems 
appropriate  to  conclude  that: 

There  is  a  need  for  further  investigation  of  the 
models  that  may  be  used  to  select  tasks  for  training. 

DEVELOPMENT  OF  TRAINING  STRATEGIES 

During  the  Concept  Phase  of  the  materiel  development  process, 
estimates  are  made  of  the  personnel,  by  MOS  and  skill  level,  required 
to  operate  and  maintain  the  new  system.  The  tasks  selected  for 
training  are  then  allocated  among  these  proposed  operator  and  main- 


tenance  positions,  and  a  training  plan  outline  is  prepared  for  each 
duty  position.  These  training  plans  contain  little  detail.  Rather, 
they  depict  the  training  strategy  by  which  personnel  resources  will  be 
developed.  These  strategies  contain  estimates  of  training  settings, 
training  media,  need  for  training  devices,  need  for  SPA  material,  and 
so  on. 

Considerable  guidance  exists  as  to  how  to  make  such  estimates 
once  detailed  task  and  skill  analysis  data  are  available.  However, 
such  guidance  documents  seldom  discuss  the  application  of  these  tech¬ 
niques  during  the  Concept  Phase.  For  example,  TRADOC  Pamphlet  350-30 
and  TRADOC  Pamphlet  351-4  both  review  procedures  for  selecting 
instructional  settings,  but  the  readers  of  these  documents  are  on 
their  own  when  applying  these  procedures  to  new  materiel.  Similarly, 
numerous  procedures  exist  for  selecting  media  but  these  procedures  do 
not  address  media  selection  during  the  Concept  Phase  of  materiel  deve¬ 
lopment.  In  fact,  a  recent  report  (Honeywell,  1978)  suggests  that 
media  selection  techniques  in  general  need  considerable  improvement. 

Recent  and  on-going  studies  have  been  concerned  with  improving 
guidance  on  designing  training  devices.  When  training  devices  are 
first  considered  in  the  Concept  Phase,  the  question  is  not  how  to 
design  them  but  rather  whether  they  are  needed  at  all.  The  problem  is 
one  of  deciding  the  strategy  for  using  training  aids  and  devices. 

This  involves  making  tentative  selections  among  actual  equipment 
trainers,  simulators,  mock-ups,  training  aids,  and  so  on.  Further,  it 
involves  estimating  training  device  requirements  in  terms  of  the  stage 
of  learning  that  the  device  is  to  support.  The  relationship  between 
stages  of  learning  and  training  devices  requirements  has  not  been 
fully  explored.  However,  an  article  by  Miller  (1974)  summarizes  many  of 
the  practical  findings  in  this  area.  < 

According  to  Meister  (1973)  there  is  need  for  a  task  taxonomy  that 
would  say:  Given  this  task,  equipment  should  be  designed  this  way  and 
training  should  proceed  as  follows.  The  considerable  research  that 
has  gone  into  development  of  task  taxonomy,  plus  their  conspicuous 
non-use,  makes  one  wonder  whether  a  useful  taxonoiny  relating  tasks  to 
training  strategies  can  be  developed. 

A  favorable  indication  comes  from  the  observation  that 
experienced  training  developers  can  look  at  many  operator  and  main¬ 
tenance  tasks  and  readily  describe  one  or  two  strategies  for  teaching 
that  task.  The  major  tasks  performed  by  maintenance  personnel  already 
have  been  classified  into  a  small  number  of  categories--! nspect, 
troubleshoot,  adjust,  remove/replace,  etc.  The  development  of  two  or 
three  different  strategies  for  teaching  each  of  these  groups  of  tasks 
should  assist  the  preparation  of  an  OICTP  during  the  Concept  Phase. 
Similar  groupings  could  be  developed  for  operator  tasks  followed  by 
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the  development  of  training  strategies  for  each.  With  respect  to  a 
technological  gap  that  seems  capable  of  being  successfully  treated: 

There  is  a  need  for  procedures,  based  on  a  task 
taxonomy  approach,  that  would  allow  one  to  iden¬ 
tify  acceptable  alternative  training  strategies 
for  common  operator  and  maintenance  tasks.  These 
procedures  should  address  the  additional  factors 
that  should  be  used  to  select  from  among  the 
alternatives. 


TECHNOLOGICAL  GAPS— COST  AND  TRAINING  EFFECTIVENESS  ANALYSIS 

More  research  is  needed  to  provide  guidance  on  applying  cost- 
effectiveness  techniques  to  the  process  of  selecting  training  sub¬ 
systems.  TRADOC  Pamphlet  71-10,  no  longer  in  force,  provides  a 
good  description  of  various  types  of  Cost  and  Training  Effectiveness 
Analyses  (CTEA)  and  some  guidance  on  the  general  methodological 
approaches  required  by  each  type.  Two  on-going  ARI-sponsored 
studies1*^  each  will  prepare  a  guidance  document  describing  how  to 
conduct  a  CTEA.  Also,  a  new  regulation  covering  CTEAs  is  being  pre¬ 
pared  by  the  Army's  TRADOC  System  Analysis  Agency  (TRASANA). 

At  present  there  is  a  tendency,  created  by  the  lack  of  specific 
guidance,  to  satisfy  the  CTEA  requirement  by  showing  that  the  use  of 
simulators  is  less  costly  than  the  use  of  actual  equipment  for  train¬ 
ing  operator  and/or  maintenance  personnel.  Although  this  procedure 
is  often  necessary,  it  may  not  be  adequate.  Other  alternatives  to  the 
simulators  may  not  have  been  evaluated  and  thus  a  "least  cost"  training 
solution  was  not  reached.  This  tendency  toward  the  cost-effective  eva¬ 
luation  of  training  devices  rather  than  of  the  entire  training  subsystem 
is  caused  by: 

(1)  A  lack  of  basic  data  needed  to  perform  such  evaluations. 

(2)  A  lack  of  comprehensive  methodologies  to  apply  to  the  various 
types  of  CTEAs. 

(3)  A  lack  of  guidance  for  determining  which  type  of  CTEA  and 
which  methodology  to  use  at  a  particular  point  in  the  LCSMM. 


^Development  of  a  Cost/Training  Effectiveness  Manual  for  Use  in 
LCSMM,  ARI,  Fort  Bliss. 

Effectiveness  of  Infantry  Systems:  TEA,  CTEA,  and  Human  Factors 
in  System  Development  and  Fielding. 
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Admittedly  this  is  a  sequentia"  type  of  problem;  methodologies 
are  not  developed  because  basic  data  are  not  available,  and  guidance 
is  not  necessary  because  the  methodologies  are  not  developed.  The 
problem  is  certainly  not  being  ignored.  Several  on-going  research 
efforts  at  ARI  are  exploring  various  aspects,  and  TRASANA  is  preparing 
to  provide  technical  assistance  in  the  performance  of  CTEAs  for 
systems  in  the  future.  These  efforts  will  undoubtedly  improve  the 
CTEA  process.  What  is  necessary  is  a  comprehensive  plan  for  incor¬ 
porating  the  improvements  achieved  into  the  LCSMM. 

The  greatest  shortfall  in  this  area  is  the  lack  of  basic  data  in 
the  early  (conceptual)  phase  of  system  development.  The  training  sub¬ 
system,  and  possibly  several  alternatives,  must  be  postulated  from  a 
list  of  critical  tasks  provided  by  the  materiel  developers.  There  are 
methods  for  accomplishing  this  process,  some  described  in  the  accom¬ 
panying  handbook  (11),  but  no  significant  basic  data.  The  entire  process 
is  essentially  a  judgment  call.  It  should  be  possible  to  compile  our 
previous  knowledge  of  system  development  and  associated  training  sub¬ 
system  development  in  such  a  way  as  to  make  the  process  of  postulating 
a  new  training  subsystem  more  deductive.  As  an  example  a  handbook 
could  be  prepared  in  the  fashion  of  weapons  effectiveness  handbooks, 
providing  the  basic  relationships  needed  to  determine  training  strate¬ 
gies  in  a  simple  and  refinable  format. 

Cost  data  for  CTEAs  are  more  readily  available  but  often  are 
located  in  the  files  of  specific  users  rather  than  being  disseminated 
for  general  use.  A  compendium  of  the  latest  cost  information  and  com¬ 
putations  techniques,  providing  the  cost  of  additional  weeks  of 
training,  identifying  the  cost  elements  and  sources  of  latest  infor¬ 
mation  on  each,  displaying  unit  training  costs,  etc.,  would  be  of 
great  value  to  the  CTEA  study  proponent. 

If  the  above  basic  data  were  available,  the  necessary  methodolo¬ 
gies  would  evolve.  In  fact,  some  have.  Approaches,  some  automated, 
have  been  developed  for  such  areas  as  cost-effective  selection  of 
training  media.  Their  use,  however,  is  predicated  on  the  availabi¬ 
lity  of  basic  data,  such  as  which  media  approach  provides  higher  stu¬ 
dent  performance  on  each  of  the  critical  tasks.  Such  decisions, 
particularly  in  the  Concept  Phase,  are  at  present  subjective,  thus 
degrading  the  results  of  the  media  selection  methodology.  Methodology 
development  should  continue,  but  its  need  for  basic  data--which  do  not 
exist— should  be  considered,  and  the  necessary  basic  research  per¬ 
formed  to  provide  the  data. 

The  guidance  problem  also  requires  some  research.  TRADOC 
Pamphlet  71-10  describes  three  types  of  CTEAs— Train-up  Study,  (TUS), 
Training  Analysis  Cost  and  Operational  Effective  Analysis  (TAC),  and 
Training  Developments  Study  (TDS).  Although  this  document  is  no 
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longer  in  force,  the  descriptions  of  CTEA  types  and  the  requirements 
of  each  type  remain  pertinent.  Each  may  be  a  screening  CTEA  (quick- 
look)  that  determines  whether  further  analysis  is  necessary.  If  it 
is,  the  TUS  and  the  TOS  are  performed  as  stand-alone  efforts  and  the 
TAC  is  provided  as  input  for  the  COEA.  The  problem  is  in  determining 
where  in  the  LCSMM  each  type  of  study  should  be  performed,  if  needed, 
and  what  methodology  to  use  with  the  data  available  at  the  time. 


TECHNOLOGICAL  GAPS— CODIFICATION  OF  HUMAN  RESOURCES  DATA 


Training  and  human  factors  personnel  are  quick  to  criticize  engi¬ 
neers  for  not  employing  human  resources  data  during  the  design  of 
materiel.  As  already  noted,  however,  there  is  a  lack  of  data 
describing  how  equipment  should  be  designed  to  accommodate  various 
types  of  persons  or  to  foster  proficient  performance  on  various  types 
of  tasks.  Moreover,  as  Meister  has  observed  (1973),  very  few  techniques 
are  available  for  describing  human  resources  data  in  terms  that  are 
meaningful  to  the  design  engineer. 

Experience  with  system  development  and  with  trade-off  studies 
over  the  years  has  identified  the  need  for  human  resources  data  to 
show  the  relationship  of  personnel/design/training/costs,  generalized 
from  system  experience  and  research,  in  the  same  way  that  human  engi¬ 
neering  data  have  been  displayed.  There  is  a  need  for  handbooks  con¬ 
taining  charts,  graphs,  tables,  etc.  which  show  the  relationship 
between  hardware  characteristics  and  training  time,  personnel  charac¬ 
teristics  and  training,  numbers  of  personnel  vs.  costs,  and  s,  on. 

Such  handbooks  of  data  packages  could  be  used  by  training  developers 
as  they  develop  data  for  cost  trade-off  studies. 

Persons  in  the  human  resources  community  are  inherently  reluctant 
to  make  general  estimates  of  the  relationships  between  factors  unless 
these  relationships  have  been  confirmed  under  controlled  conditions. 
However,  time  and  time  again  these  same  persons  are  asked  to  make 
estimates  about  the  quantities  and  types  of  persons  needed  to  perform 
certain  tasks,  or  about  the  most  effective  form(s)  of  training 
programs  for  teaching  various  tasks  or  various  types  of  students. 

Anecdotal  evidence  suggests  that  experienced  personnel  can  make 
fairly  accurate  estimates  of  human  resources  requirements  for  various 
types  of  proposed  materiel,  especially  if  the  proposed  materiel  is  an 
advanced  version  of  something  that  exists  already.  Studies  for  the 
Air  Force's  Human  Resources  Laboratory  (Whalen  and  Askren,  1974)  have 
shown  that  technicians  can  make  accurate  predictions  about  the  parame¬ 
ters  of  general  types  of  human  resources  data  during  the  conceptual 
design  phase.  As  an  illustration,  experienced  personnel  can  quite 
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accurately  estimate  the  average  time  required  to  repair  various  equip¬ 
ment  subsystems  and  the  size  and  composition  of  flight  line  repair 
crews. 

Without  attempting  to  base  the  relationship  of  functions  on 
actual  data,  the  data  form  of  the  relationship  for  various  functions 
which  enter  into  materiel -personnel /training  trade-off  studies  may  be 
depicted  as  shown  in  Figures  2  and  3. 

Figure  2  illustrates  the  probable  relationship  between  training 
time  and  aptitude  scores.  Training  time  might  be  shown  in  hours, 
weeks,  or  months  and  aptitude  scores  expressed  in  terms  of  clerical, 
electronic  or  mechanical  aptitude  with  different  charts  prepared  for 
different  aptitudes.  Such  charts  should  be  useful  for  determining  the 
cost-effectiveness  of  certain  approaches.  For  example,  one  could  com¬ 
pare  an  approach  that  used  large  numbers  of  less  capable  persons  with 
an  approach  that  used  fewer  persons  of  a  higher  capability. 

Figure  3  illustrates  the  probable  relationship  between  maintenance 
man-hour  requirements  and  the  number  of  replaceable  parts  in  the 
system.  Parameters  X,  Y  and  Z  represent  average  materiel  reliability 
specifications.  From  such  a  chart,  one  could  determine  that  the 
number  of  maintenance  personnel  required  for  a  system  is  related  to 
the  number  of  parts  that  can  fail  (replaceable  parts)  and  the  reliabi¬ 
lity  criterion  for  each  part. 

Many  operator  and  maintenance  tasks  are  quite  similar  across 
materiel  systems.  For  each  of  these  tasks  two  or  more  equally  accep¬ 
table  training  strategies  usually  can  be  developed.  However,  these 
strategies  may  differ  widely  in  terms  of  support  requirements.  It 
would  seem  possible,  for  such  common  tasks,  to  outline  a  set  of 
training  strategies  for  each  and  describe  the  implication  of  each 
strategy  in  terms  of  factors  such  as  materiel  training  device,  and 
instructor  requirements.  This  information  then  could  be  used  during 
trade-off  studies  and  during  Cost  and  Training  Effectiveness  Analysis 
studies. 

In  view  of  the  foregoing  considerations  the  authors  believe  that: 

Charts,  graphs,  tables,  and  other  forms  of  handbook 
material  should  be  prepared  to  summarize  professional 
judgments  about  the  relationships  among:  (a)  equipment 
characteristics  and  personnel /training  requirements; 

(b)  task  requirements  and  materiel  and/or  operational 
requirements;  (c)  task  requirements  and  work  demand 
conditions;  and  (d)  task  requirements  and  probable  time 
and/or  cost  to  train. 
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Figure  2.  Training  Time  vs.  Aptitude  Scores 
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Figure  3.  Man  Hours  Required  vs.  Number  of  Parts 
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SOME  CONCERNS  OF  TRADOC  SYSTEM  MANAGERS  (TSMs) 


As  part  of  this  project  TRADOC  System  Managers  (TSMs)  and  their 
staffs,  and  other  support  personnel  were  interviewed  for  the  pur¬ 
pose  of  identifying  their  information  requirements.  TSMs  located  at 
Forts  Bliss  and  Sill  were  contacted,  as  were  persons  associated  with 
the  Training  Development  and  Course  Development  Directorates  at  these 
two  schools.  Personnel  attached  to  the  Army  Research  Institute's 
Field  Units  at  these  posts  also  were  queried.  All  were  questioned 
regarding  their  activities  relative  to  preparation  of  training 
programs  for  developing  systems,  the  sources  of  information  they  use 
when  engaged  in  such  activities,  their  use  of  existing  TSM  handbooks, 
and  their  need  for  additional  information  concerning  the  developing  of 
training  subsystems  for  new  materiel. 

Based  on  these  interviews,  26  problem  areas  were  identified. 

These  areas  are  listed  in  Table  1  and  described  in  more  detail  in 
Appendix  A.  The  interviewees  also  identified  cerain  topics  and  con¬ 
tent  that  might  be  included  in  future  handbooks  for  TSMs.  These  are 
listed  in  Table  2  and  described  more  fully  in  Appendix  B.  The 
interviewer's  findings  are  summarized  in  the  following  paragraphs. 

Currently  most  new  systems  are  being  developed  on  an  accelerated 
schedule,  and  there  is  a  concerted  effort  to  reduce  the  entire  deve¬ 
lopmental  cycle  to  about  five  years.  This  is  being  attempted  through 
the  elimination  of  DT/OT  III,  and  the  complete  or  at  least  partial 
elimination  of  DT/OT  I.  Further,  most  of  the  studies,  documents, 
training  materials,  and  training  equipment  which  should  be  developed 
prior  to  DT/OT  II  are  being  prepared  by  the  prime  contractor(s).  As  a 
consequence,  it  appears  that  many  of  the  training  and  human  factors  con¬ 
siderations  which  should  be  addressed  during  Phases  I  and  II  of  the 
LCSMM  are  receiving,  or  will  receive,  minimum  attention.  It  seems 
probable,  therefore,  that  many  of  the  training  system  development 
problems  which  the  Integrated  Personnel  Support  procedures  were 
designed  to  alleviate  will  instead  be  exacerbated  by  efforts  to  reduce 
and/or  bypass  major  portions  of  the  training  system  development  pro¬ 
cess. 


The  interviewers  were  impressed  with  how  much  of  the  integrated 
personnel  support  system  for  new  systems  is  contractor  developed. 

This  is  now  the  rule,  rather  than  the  exception.  Neither  TSM  offices 
nor  School  Directorates  have  the  personnel  needed  to  prepare  the 
various  documents  and  conduct  the  various  studies  called  for  in  TRADOC 
Regulation  600-4.  In  the  typical  case,  Individual  and  Collective 
Training  Plans,  Training  Device  Requirement  studies,  provisional 
QQPRIs,  training  materials,  and  training  devices  are  developed  under 
contract.  Currently,  the  critical  task  of  TSMs  and  other  Army  person¬ 
nel  is  to  monitor  and  evaluate  contractor  activities  and  products, 
and  guidance  for  doing  this  needs  considerable  improvement. 
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Numerous  TSMs  expressed  the  need  for  "backward  planning"  guidance. 
They  desire  a  document  that  will  help  them  plan  backwards  from  a 
scheduled  ASARC/DSARC  review  or  an  Operational  Test  date.  This  guide 
should  describe  the  documents  and  materials  which  must  be  prepared 
prior  to  those  scheduled  activities,  approximately  how  long  it  will  take 
to  prepare  the  documents/materials,  and  who  is  responsible  for  pre¬ 
paring  them. 

At  both  Fort  Bliss  and  Fort  Sill,  TSMs  are  located  in  adjoining 
offices,  yet  there  appeared  to  be  little  formal  effort  to  exchange 
experiences.  In  explanation  of  this  seeming  lack  of  cooperation, 
the  interviewers  were  told  that  each  new  system  is  different,  is  at  a 
different  stage  in  the  LCSMM,  and  is  following  a  somewhat  different 
time  line.  Because  of  these  considerations,  many  TSMs  were  of  the 
opinion  that  a  detailed  handbook  for  TSMs  would  have  little  overall 
value.  Some  noted,  however,  that  efforts  to  identify  exemplary  docu¬ 
ments  and  disseminate  these  as  models  to  all  TSMs  would  be  useful. 

They  also  noted  that  the  main  commonality  between  all  systems  is  the 
requirement  to  hold  scheduled  ASARC/DSARC  reviews  at  the  end  of  each 
phase  of  the  LCSMM. 

There  was  disagreement  reqarding  the  usefulness  of  handbooks 
for  TSMs.  All  TSMs  were  aware  of  the  MITRE  and  ARI/TRADOC  (Hanson 
and  Purifoy,  1978)  TSM  handbooks  but  reported  little  use  of  them. 

Some  interviewees  discussed  the  need  for  more  detailed  "how  to  do  it" 
manuals  for  TSMs,  manuals  that  would  consolidate  other  references  and 
serve  as  a  prime  source  document.  Other  interviewees  were  of  the  opi¬ 
nion  that  detailed  handbooks  get  out  of  date  too  quickly  and  do  not 
provide  for  enough  flexibility  of  action;  they  favored  the  preparation 
of  guidebooks  rather  than  "cookbooks". 
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Table  1 

PROBLEMS  IDENTIFIED  BY  TRADOC  SYSTEM  MANAGERS 
AND  ASSOCIATED  PERSONNEL 


1.  Compression  of  LCSMM  into  five  years  does  not  allow  for  enough 
time  to  concurrently  develop  a  training  system. 

2.  Human  factors  problems  receive  little  consideration  until  an 
advanced  prototype  is  available  at  OT  II. 

3.  Most  training-related  documents,  studies,  and  material  for  new 
systems  are  developed  under  contract.  This  increases  the 
monitoring/evaluating  burden  of  Army  personnel. 

4.  Documents  describing  the  LCSMM  process  are  of  only  general  use 
because  each  system  is  being  developed  under  a  different  schedule. 

5.  Front-End  Analysis  (FEA)  data  often  are  acquired  on  an  untimely 
and  incomplete  basis. 

6.  More  guidance  is  needed  regarding  how  to  state  training  system 
requirements  in  RFPs  and  in  contracts. 

7.  Better  guidance  is  needed  regarding  how  to  evaluate  statements  of 
training  device  requirements  (TDRs)  as  well  as  any  training 
devices  developed  under  contract. 

8.  Better  guidance  is  needed  regarding  how  to  monitor  and  evaluate 
contractor  efforts. 

9.  Improved  guidance  and  methods  are  needed  to  support  the 
conduct  of  CTEAs  and  COEAs. 

10.  Better  guidance  is  needed  regarding  the  staffing  of  the 
various  documents  which  must  be  produced  during  the  LCSMM. 

11.  TSM  authority  to  task  other  TRADOC  agencies  is  not  clear. 

12.  More  information  about  budgeting  cycles  and  related  lead  times 
is  needed  by  most  TSMs. 

13.  Improved  guidance  is  needed  regarding  how  to  prepare  for  ASARC/ 
DSARC  reviews. 
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Table  1  (Continued) 

14.  Better  guidance  is  needed  regarding  how  to  "backward  plan"  from 
LCSMM  milestones. 

15.  Improved  guidance  is  needed  regarding  the  format  and  contents  of 
LCSMM-related  documents. 

16.  No  mechanism  exists  for  identifying  exemplary  documents  and  inform¬ 
ing  TSMs  about  such  documents. 

17.  No  formal  procedures  exist  for  sharing  information  among  tsms. 

18.  More  guidance  is  needed  regarding  how  to  manage  the  development 
of  systems  under  the  Rationalization,  Supportabil ity. 
Interoperability  (RSI)  Program. 

19.  Some  LCSMM-related  documents  that  are  not  now  action  documents 
should  be  so  designated. 

20.  Inaccuracies  in  Reliability,  Availability,  Maintainability  (RAM) 
data  can  lead  to  conflicts  regarding  QQPRIs. 

21.  Better  and  less  cumbersome  media  selection  techniques  are  needed. 

22.  Validation  of  SPA  material  sometimes  does  not  occur  until  after 
0T  II. 

23.  Logistics  agencies  are  sometimes  slow  to  provide  inputs  to  TSMs. 

24.  The  definition  of  Initial  Operational  Capability  (IOC)  is  unclear. 

25.  Persons  assigned  to  a  TSM's  staff  should  be  familiar  with 
the  general  developmental  cycle  for  new  equipment. 

26.  No  satisfactory  procedures  exist  for  preserving  the  "institutional 
experience"  developed  within  a  TSM  office. 
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Table  2 


SUGGESTED  TOPICS/CONTENT  FOR  TSM  HANDBOOK 


1.  Backward  planning  guidance 

2.  Proponent  planning  guidance 

3.  Information  about  the  format  and  content  requirements  of  key 
documents 

4.  Guidance  regarding  the  preparation/ re view  of  RFPs  and  contracts 

5.  Guidance  regarding  how  to  monitor  contractor  efforts  and  to 

evaluate  contract  deliverables 

6.  Information  describing  TSM  involvement  with  Operational  Testing 

7.  Guidance  regarding  the  role  of  other  TRADOC  agencies 

8.  Guidance  regarding  the  conduct  of  CTEAs 

9.  Guidance  regarding  the  Rationalization,  Supportabil ity. 

Interoperability  (RSI)  program 

10.  Guidance  regarding  how  to  assure  consideration  of  human  factors 
during  early  phases  of  the  LCSMM 
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SUMMARY  AND  CONCLUSIONS 


The  Life  Cycle  System  Management  Model  (LCSMM)  outlines  an 
orderly  sequence  of  steps  that  must  be  accomplished  during  the 
development  of  new  materiel  systems  for  the  Anry.  The  Integrated 
Personnel  System  (IPS)  model  describes  a  related  series  of  steps  that 
should  be  followed  to  ensure  that  cost-effective  and  timely  training 
and  personnel  subsystems  are  developed  and  tested  concurrent  with 
materiel  development.  It  is  anticipated  that  systems  developed  under 
these  two  models  will  be  developed  more  rapidly,  will  be  more  cost 
effective,  will  have  more  effective  support  subsystems,  and  will 
require  less  retrofitting  once  they  become  operational. 

The  findings  of  this  report  suggest  that  the  gains  expected  through 
application  of  the  foregoing  models  will  not  be  achieved  because  the 
techniques  for  executing  various  steps  in  the  models  exist  only  in 
rudimentary  form,  if  at  all.  This  conclusion  is  based  on  the  exis¬ 
tence  of  three  categories  of  technological  gaps  which  prevent  adequate 
development  of  personnel  and  training  subsystems  until  an  advanced 
prototype  of  system  hardware  has  been  produced.  These  gaps  concern: 

(1)  The  lack  of  techniques  for  generating  needed  data  at 
various  points  in  the  LCSMM  development  process. 

(2)  The  lack  of  techniques  for  predicting  the  impact  of 
materiel  design  on  training/ personnel  subsystems,  and 
conversely,  the  impact  of  these  subsystems  on  materiel 
design. 

(3)  The  difficulty  of  developing  defensible  training  plans 
and  cost-effectiveness  estimates  during  the  early  stages 
of  materiel  development. 

The  existence  of  these  and  related  technological  gaps  has  been 
recognized  for  some  time,  and  efforts  are  underway  to  fill  these  gaps 
with  improved  techniques.  These  efforts  must  continue,  if  the  objec¬ 
tive  of  fielding  new  materiel  systems  with  well -trained  personnel  is 
to  be  achieved. 

During  Phase  I  of  the  LCSMM,  alternative  materiel  concepts  are 
investigated,  and  for  each  concept  personnel  and  training  requirements 
must  be  estimated.  Development  of  these  estimates  is  hindered  by  the 
absence  of  satisfactory  techniques  for: 

(1)  Deducing  training  and  personnel  requirements  from 
materiel  characteristics. 

(2}  Identifying  excessive  human  resources  demands  stemming 
from  materiel  concepts. 


(3)  Identifying  functions  most  appropriately  performed  by 

equipment,  by  persons,  and  by  some  man-machine  combinations. 

During  this  Concept  Phase  of  the  LCSMM,  the  cost  effectiveness  of 
alternative  training  plans  must  be  assessed.  This  process  depends  on 
the  development  of  task  and  skill  data,  the  identification  of  high- 
risk  training  tasks,  and  the  development  of  training  strategies, 
existing  Instruction  System  Development  techniques  and  task  analytic 
techniques  for  accomplishing  such  activities  assume  that  hardware 
exists.  However,  during  the  Concept  Phase  for  developing  systems  pro¬ 
totype  hardware  does  not  exist.  Thus,  important  personnel  and  training 
decisions  now  must  be  based  on  guesswork  or  on  data  from  comparable 
systems  or  both.  There  is  a  need,  therefore,  to  devise  techniques 
that  can  be  applied  during  the  Concept  Phase  of  developing  systems  to 
generate  task  analytic  data,  identifying  high-risk  training  tasks,  and 
indicate  appropriate  training  strategies  before  the  actual  hardware 
system  is  in  existence. 

For  future  materiel  systems  it  is  quite  possible  that  training 
and  personnel  support  subsystems  will  be  more  costly  than  their 
materiel  counterpart.  Therefore,  better  techniques  are  needed  for 
developing  estimates  of  training  costs  and  training  effectiveness  for 
alternative  training  strategies.  Currently,  baseline  data  upon  which  to 
conduct  cost  studies  of  training  and  operational  effectiveness  are 
usually  lacking  during  the  LCSMM  Concept  Phase.  Also,  there  is  an 
absence  of  comprehensive  and  standardized  methodologies  for  accomplish¬ 
ing  such  st  Jies.  On-going  ARI-sponsored  research  may  correct  this 
technological  gap. 

Human  factors  and  training  personnel  often  complain  about  the 
lack  of  attention  given  to  human  resources  considerations  during 
materiel  design.  Air  Force  studies  have  documented  the  legitimacy  of 
such  complaints.  However,  many  authors  have  noted  the  almost  total 
absence  of  techniques  for  deducing  materiel  characteristics  from  human 
resources  constraints.  Techniques  do  not  exist  for  describing  human 
resources  data  and/or  requirements  in  terms  that  are  meaningful  to 
design  engineers.  Until  this  technological  gap  is  filled,  it  is  un¬ 
likely  that  materiel  will  be  designed  to  meet  anticipated  constraints 
in  human  resources. 

In  the  past,  new  systems  were  designed  to  meet  certain  mission  or 
functional  requirements,  and  it  was  assumed  that,  through  selection 
and  training,  the  human  components  of  the  new  system  could  compensate 
for  deficiencies  in  system  hardware  or  for  functional  requirements 
that  could  not  be  met  by  the  hardware  portion  of  the  system.  This 
assumption  is  no  longer  tenable.  The  high  cost  of  future  training  and 
personnel  systems,  plus  the  anticipated  shortfall  of  persons  having 
appropriate  capabilities,  makes  it  Imperative  that  future  systems  be 
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designed  to  minimize  training  and  personnel  costs,  while  at  the  same 
time  minimizing  the  need  for  exceptional  persohhel. 
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APPENDIX  A 

Problem  Areas  Identified  by  TRADOC  System  Managers  (TSMs) 
and  Other  Training  Proponent  Personnel 


1.  Compression- of  LCSMM  into  five  years  does  not  allow  for  enough 

time  to  concurrently  develop  a  training  system. 

Most  new  systems  are  being  developed  under  an  accelerated  sche¬ 
dule.  OT  III  is  being  eliminated,  and  in  some  instances  OT  I  is  being 
slighted  or  bypassed.  When  OT  I  is  bypassed,  competing  contractors 
develop  prototype  equipment  which  is  first  evaluated  during  a  test 
similar  to  OT  II.  The  results  of  this  evaluation,  along  with  contrac¬ 
tor-developed  proposals  for  training  systems,  are  used  to  select  the 
winning  system.  Limited  production  then  will  begin.  The  foregoing 
developmental  schedule  probably  will  result  in  the  deemphasis  of 
training  and  human  factors  considerations  during  the  first  two  phases 
of  the  LCSMM.  It  seems  probable  also  that,  because  of  competitive 
reasons,  contractors  will  be  reluctant  to  allow  Army  personnel  to 
monitor  their  contractual  efforts  prior  to  the  initial  evaluation  of 
the  competing  systems. 


2.  Human  factors  problems  receive  little  consideration  until  an  advanced 

prototype  is  available  at  OT  II. 

Potential  human  factors  problems  should  receive  initial  con¬ 
sideration  during  Phase  I  of  the  LCSMM,  and  should  receive  extensive 
consideration  during  Phase  II.  In  practice,  human  factors  problems 
receive  little  considerat *0*.  until  after  OT  II.  For  example,  OT  I  or 
its  equivalent  may  be  based  on  an  evaluation  of  the  feasibility  of 
available  technology  or  on  an  evaluation  of  unintegrated  portions  of 
the  proposed  system.  It  is  difficult  to  identify  potential  human  fac¬ 
tors  problems  during  such  types  of  evaluation.  Thus,  human  factors 
problems  may  not  be  identified  until  OT  II.  By  this  time  it  is  dif¬ 
ficult  to  effect  major  changes  in  the  hardware.  This  is  likely  to  happen 
for  those  systems  being  developed  under  the  Rationalization,  Support- 
ability,  Interoperability  (RSI)  program.  Under  this  program  all 
equipment  changes  must  be  coordinated  with  and  agreed  to  by  allied 
representatives.  For  some  equipment  provision  has  been  made  for 
early  inputs  to  Human  Engineering  Laboratory  (HEL).  However,  these 
inputs  often  are  made  after  OT  I. 
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3.  Most  training-related  documents,  studies,  and  material  for  new 
systems  are  developed  under  contract.  This  increases  the 
monitoring/evaluating  burden  of  Army  personnel. 

The  current  trend  is  to  contract  out  the  development  of  as  much 
of  the  training  system  as  possible.  This  means  that  Army  employees 
will  be  used  increasingly  to  monitor  and  to  evaluate  the  development 
of  training  materials.  Guidance  regarding  how  to  accomplish  this 
needs  considerable  improvment.  Currently  one  can  refer  to  various 
Military  Standards  to  determine  whether  SPA  materials  meet  various 
requirements.  However,  there  are  no  standards  for  evaluating  training 
devices  or  such  documents  as  Individual  and  Collective  Training  Plans 
(ICTPs),  provisional  QQPRIs,  and  so  on. 


4.  Documents  describing  the  LCSMM  process  are  of  only  general  use 

because  each  system  is  being  developed  under  a  different  schedule. 

The  development  schedule  for  a  new  system  depends  on  how  dif¬ 
ferent  it  is  from  current  systems,  recent  re-evaluation  of  the  threat 
which  the  system  is  designed  to  meet,  recent  technological  advances, 
and  so  on.  Furthermore,  many  systems  now  under  development  were  begun 
before  current  LCSMM  procedures  were  established.  Thus,  the  time 
schedule  for  each  system  is  different  (and  determined  primarily  by  the 
Project  Manager);  certain  key  documents  required  by  the  LCSMM  are  pre¬ 
pared  on  a  retroactive  basis;  and  certain  key  steps  in  the  LCSMM  are 
bypassed  on  a  "management  by  exception"  basis.  For  these  and  other 
reasons  TSMs  do  not  pay  much  attention  to  the  details  of  the  LCSMM. 

What  they  attend  to  are  scheduled  ASARC/DSARC  reviews.  The  schedule 
for  these  reviews  plus  the  scheduled  time  of  IOC  is  what  drives  the 
entire  developmental  cycle. 


5.  Front-End  Analysis  (FEA)  data  often  are  acquired  on  an  untimely 

and  incomplete  basis. 

Front-End  Analysis  data  supposedly  are  supplied  initially 
following  award  of  the  Advanced  Development  contract  and  are  updated 
after  each  Operational  Test.  These  data  are  provided  by  the  contractor. 
Typically,  they  are  not  provided  until  after  OT  I  and  sometimes 
not. until  after  OT  II.  The  Army  finds  that  FEA  data  provided  for  the 
purpose  of  developing  a  Preliminary  ICTP,  QQPRI,  Test  Plan  for  OT  I, 
etc.,  usually  are  late  in  arriving  and  often  are  incomplete.  This  may 
be  a  contractual  problem  which  can  be  alleviated  by  more  specific 
contractual  guidance  and  requirements.  If  this  practice  is  to  con¬ 
tinue,  the  Army  must  be  allowed  to  monitor  the  development  of  these 
plans  and  documents  in  order  to  synchronize  other  subsystem 
development. 
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6.  More  guidance  is  needed  regarding  how  to  state  training  system 

requirements  in  RFPs  and  in  contracts. 

TSMs  and  persons  associated  with  school  directorates  often  are 
involved  in  the  preparation  of,  or  at  least  the  review  of,  RFPs  and 
contracts.  The  preparation  of  these  documents  for  new  systems  is  of 
critical  importance  because  contractors  are  asked  to  produce  many 
of  the  products  which  heretofore  were  developed  by  Army  employees.  It 
is  especially  important  that  the  contract  be  written  so  that  military 
personnel  will  have  the  opportunity  to  closely  monitor  the  development 
of  documents  and  material  related  to  the  training  system.  MIL-STD- 
1388  (14)  describes  how  SPA  requirements  should  be  written  into  an 
RFP.  Interviewees  expressed  a  need  for  better  guidance  in  other 
areas. 


7.  Better  guidance  is  needed  regarding  how  to  evaluate  statements  of 
training  device  requirements  (TDRs)  as  well  as  any  training  devices 
developed  under  contract. 

According  to  the  LCSMM,  training  device  requirements  should  be 
identified  in  conjunction  with  the  development  of  a  preliminary 
Individual  and  Collective  Training  Plan.  The  guidance  implies 
that  TDRs  should  be  developed  by  Army  personnel.  In  practice  the 
requirement  to  identify  TDRs  often  is  incorporated  into  an  RFP  or  a 
Required  Operational  Capability  (ROC)  document.  In  such  cases  the 
contractor  has  the  responsibility  for  both  identifying  the  need  for 
training  devices  and  developing  such  devices.  The  responsibility  of 
Army  personnel  is  to  evaluate  any  statement  of  TDRs  and  any  devices 
developed  by  the  contractor.  Better  guidance  is  needed  on  how  to 
accomplish  this. 


8.  Better  guidance  is  needed  regarding  how  to  monitor  and  evaluate 
contractor  efforts. 

As  noted  already,  most  of  the  documents  and  deliverables  asso¬ 
ciated  with  the  LCSMM  are  produced  under  contract.  The  Army  is 
responsible  for  monitoring  the  development  of  these  products  and  for 
their  evaluation.  Numerous  interviewees  reported  that  there  is  little 
guidance  in  these  areas,  and  that  the  development  of  improved  guidance 
would  be  useful. 
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9.  Improved  guidance  and  methods  are  needed  to  support  the 
conduct  of  CTEAs  and  COEAs. 

The  interviewees  reported  that  little  guidance  exists  regarding 
how  to  conduct  a  Cost  and  Training  Effectiveness  Analysis  (CTEA).  In 
addition  to  general  guidance,  the  interviewees  desired  more 
information  on  detailed  methods  for  accomplishing  CTEAs.  This  need 
already  has  been  recognized  and  is  the  subject  of  on-going,  ARI- 
sponsored  research. 

10.  Better  guidance  is  needed  regarding  the  staffing  of  the 
various  documents  which  must  be  produced  during  the  LCSMM. 

TSM  personnel  did  not  agree  as  to  the  need  for  more  detailed 
guidance  regarding  the  staffing  of  various  documents  and  studies. 
However,  some  TSMs  and/or  their  staffs,  reported  that  for  certain 
documents  the  staffing  guidance  was  not  clear. 


11.  TSM  authority  to  task  other  TRADOC  agencies  is  not  clear. 

To  accomplish  their  mission  TSMs  are  authorized  to  interact 
directly  with  other  Army  agencies.  In  most  instances  the  nature  of 
this  interaction  is  spelled  out  rather  well  in  regulations.  However, 
TSMs  also  are  authorized  to  use  the  services  of  other  TRADOC  agencies. 
The  nature  of  this  authority  apparently  is  not  as  clear  as  it  might 
be.  Some  TSMs  expressed  a  desire  to  know  more  about  their  authority 
to  task  other  TRADOC  agencies,  what  these  other  agencies  could  do  for 
them,  the  procedures  for  gaining  their  assistance,  and  the  appropriate 
entry  points  to  these  agencies. 


12.  More  information  about  budgeting  cycles  and  related  lead  times 

is  needed  by  most  TSMs. 

TSMs  continually  discover  that  they  cannot  obtain  large  amounts 
of  school  support  until  the  support  requirements  have  been  placed  in 
the  school  budget.  This  takes  time.  For  example,  if  instructors  are 
required  for  OT  II,  it  may  take  up  to  29  months  to  obtain  these 
instructors  through  the  normal  budget  cycle.  If  new  facilities  are 
required  for  training,  it  usually  takes  five  years  to  obtain  these  faci¬ 
lities.  TSMs  and  their  staffs  seldom  are  aware  of  the  specific  lead- 
time  requirements  associated  with  their  various  requests.  This  type 
of  information  would  be  of  use  to  them.  The  Field  Artillery  School 
has  developed  a  budgeting  slide  rule  which  provides  this  information, 
and  has  made  It  available  to  TSMs.  Similar  devices  have  been 
developed  for  use  at  other  Army  schools. 
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13.  Improved  guidance  is  needed  regarding  how  to  prepare  for  ASARC/DSARC 

reviews. 

There  are  four  key  dates  in  the  LCSMM:  the  Initial  Operational 
Capability  (IOC)  date,  and  the  three  ASARC/DSARC  review  dates.  During 
each  ASARC/DSARC  review,  a  particular  set  of  issues  is  addressed  and  a 
particular  set  of  documents  and  other  deliverables  should  be  available. 
Numerous  TSMs  expressed  an  interest  in  having  more  definitive  guidance 
regarding  how  to  prepare  for  ASARC/DSARC  reviews.  In  particular  they 
wanted  better  guidance  regarding  the  documents  and  deliverables  that 
should  be  ready  prior  to  these  reviews.  This  is  in  keeping  with  the 
expressed  needs  of  TSMs  for  more  information  that  can  be  used  for 
backward  planning. 


14.  Better  guidance  is  needed  regarding  how  to  "backward  plan"  from 

LCSMM  milestones. 

As  previously  mentioned,  the  critical  dates  in  the  LCSMM  are 
those  associated  with  the  ASARC/DSARC  reviews  and  with  the  IOC. 
Numerous  TSMs  expressed  an  interest  in  being  provided  with  more  back¬ 
ward  planning  guidance,  especially  with  reference  to  ASARC/DSARC 
reviews.  They  would  like  to  know  what  documents  and  other  deli¬ 
verables  need  to  be  ready  at  these  points;  they  would  like  to  have 
estimates  of  the  time  required  to  develop  these  deliverables;  and  they 
would  like  to  know  who  should  be  tasked  to  develop  the  deliverables. 
The  Directorate  of  Course  Development  at  the  Field  Artillery  School 
has  developed  a  slide  rule  that  can  be  used  to  determine  lead  time 
requirements  for  accomplishment  of  various  training-related  actions. 
More  planning  guidance  of  a  similar  nature  would  seem  appropriate. 


15.  Improved  guidance  is  needed  regarding  the  format  and  contents  of 
LCSMM-related  documents. 

In  most  cases  each  major  document  required  as  part  of  the  LCSMM  is 
described  in  a  regulation.  However,  these  regulations  do  not  describe 
in  detail  the  format  of  some  of  the  required  documents  nor  the  topics 
that  should  be  addressed.  Numerous  persons  felt  that  more  guidance, 
with  examples,  would  be  useful. 
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16.  No  mechanism  exists  for  identifying  exemplary  documents  and  informing 
TSMs  about  such  documents. 

No  formal  means  have  been  established  for  identifying  good  examples 
of  LCSMM-related  documents  and  disseminating  this  information  to  all 
TSMs.  The  TACFIRE  ICTP  has  been  labeled  as  a  "GOOD"  example  of  an 
ICTP  and  has  been  distributed  to  most  TSMs.  It  was  suggested  that  the 
TRADOC  TSM  Office  might  try  to  identify  other  exemplary  documents  and 
provide  these  to  all  TSMs. 


17.  No  formal  procedures  exist  for  sharing  information  among  TSMs. 

As  noted  previously,  there  are  no  formal  mechanisms  for  sharing 
information  among  TSMs.  Most  TSMs  were  of  the  opinion  that  some  sort 
of  information  sharing  should  be  useful,  and  suggested  that  the  TRADOC 
TSM  office  might  assume  responsibility  for  such  actions. 


18.  More  guidance  is  needed  regarding  how  to  manage  the  development 
of  systems  under  the  Rationalization,  Supportabil ity, 
Interoperability  (RSI)  program. 

Some  of  our  new  systems  are  being  developed  in  cooperation  with 
our  allies.  This  means  that  prototype  equipment  and  draft  training 
material  must  be  approved  by  a  committee  composed  of  all  concerned 
parties,  and  changes  cannot  be  made  unless  approved  by  all  parties. 
Apparently,  there  is  little  guidance  regarding  what  type  of  changes 
will  have  a  major  impact  on  the  system  and  will  require  an  agreement 
with  our  allies.  For  example,  modifying  equipment  to  correct  a  human 
factors  deficiency  may  impact  heavily  on  the  assembly  line  of  some 
German  manufacturer.  Some  TSMs  have  had  difficulty  explaining  to 
their  staff  the  restrictions  imposed  by  the  RSI  program. 


19.  Some  LCSMM-related  documents  that  are  not  now  action  documents 

should  be  so  designated. 

Some  of  the  key  documents  developed  during  the  first  and  second 
phases  of  the  LCSMM  are  not  action  documents.  In  particular,  the  pre¬ 
liminary  QQPRI  is  developed  and  then  widely  staffed  as  an  information 
copy.  A  copy  of  the  PQQPRI  is  forwarded  to  MILPERCEN  but  they  are  not 
not  required  to  respond  to  this  copy  even  if  they  are  aware  of  potential 
conflicts  between  the  proposed  QQPRI  and  personnel  recruitment  plans. 

As  a  result,  the  TSM  may  develop  a  final  QQPRI  that  is  in  conflict 
with  MILPERCEN's  perception  of  the  types  and  quantities  of  people  who 
can  be  provided  in  a  future  time  frame.  Such  disagreements  should  be 
resolved  as  early  as  possible,  otherwise  systems  that  cannot  be  ade¬ 
quately  manned  may  be  developed. 


33 


20.  Inaccuracies  in  Reliability,  Availability,  Maintainability  (RAM) 

data  can  lead  to  conflicts  regarding  QQPRIs. 

The  development  of  a  provisional  QQPRI  depends  on  the  availabi¬ 
lity  of  RAM  data.  These  data  are  obtained  from  the  records  of  equipment 
similar  to  that  proposed  for  the  new  system.  When  there  are  no  simi¬ 
lar  equipments,  RAM  data  are  developed  on  the  basis  of  professional 
judgment.  During  Phase  I  of  the  LCSMM,  available  RAM  data  are  almost 
certain  to  contain  inaccuracies,  which  can  lead  to  conflicts  between 
CDC,  TSMs,  and  MILPERCEN  regarding  the  composition  of  a  QQPRI.  For 
example,  CDC  might  estimate  that  certain  numbers  of  persons  by  MOS 
will  be  required  to  maintain  the  new  system.  Based  on  a  Task  and 
Skill  Analysis  performed  either  by  the  contractor  or  by  a  TRADOC 
school,  the  TSM  may  have  different  views  regarding  the  numbers  and 
types  of  persons  required  to  man  the  system.  To  complicate  matters 
further,  MILPERCEN  may  claim  that,  based  on  its  recruitment  projec¬ 
tions,  it  will  not  be  possible  to  provide  the  persons  required  by 
either  the  CDC-  or  TSM-proposed  QQPRI.  Certain  systems  are  now 
approaching  their  final  stage  of  development  without  having  an  agreed- 
upon  QQPRI. 

21.  Better  and  less  cumbersome  media  selection  techniques  are 

needed. 

During  the  development  of  Individual  and  Collective  Training 
Plans,  and  the  identification  of  training  device  requirements, 
numerous  decisions  must  be  made  regarding  the  best  media  to  employ. 

The  interviewers  were  told  that  current  media  selection  techniques  are 
cumbersome  and  are  of  doubtful  value.  There  seems  to  be  no  standard 
way  by  which  TSMs  or  those  tasked  by  TSMs  attempt  to  handle  this 
problem.  Some  seek  assistance  from  NTDC.  Others  assign  the  problem 
to  some  group  within  the  Directorate  of  Course  Development.  For  still 
other  systems,  the  contractor  Is  responsible  for  matching  training 
requirements  and  training  media.  It  appears  that  the  media  selection 
problem  arises  in  part  because  many  people  do  not  know  how  to  employ 
existing  media  selection  techniques  and  therefore  tend  to  rely  on 
historical  precedent. 
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22.  Validation  of  SPA  material  sometimes  does  not  occur  until  after 

OT  II. 

The  need  for  SPA  material  should  be  identified  during  Phase  I  of 
the  LCSMM,  and  critical  portions  of  this  material  should  be  evaluated 
during  OT  I.  All  SPA  material  should  be  available  for  evaluation 
during  OT  II.  Apparently  this  is  seldom  the  case.  In  many  instances 
SPA  material  is  not  available  even  by  OT  II.  Also,  SPA  material, 
especially  TEC  material,  often  arrives  in  an  unsatisfactory  state  and 
has  to  undergo  extensive  and  expensive  revisions  before  it  can  be 
demonstrated  to  be  effective.  It  was  noted  also  that  there  are  no 
good  procedures  for  assessing  the  readability  level  of  figures, 
diagrams,  and  charts. 


23.  Logistics  agencies  are  sometimes  slow  to  provide  inputs  to  TSMs. 

TSMs  must  coordinate  their  activities  with  various  agencies. 
Often  this  is  done  through  the  convening  of  meetings.  Some  TSMs 
reported  that  logistics  agencies  tend  not  to  send  representatives  to 
such  meetings.  Therefore,  inputs  from  logistics  agencies  tend  to  be 
untimely  and  may  not  even  be  provided. 


24.  The  definition  of  Initial  Operational  Capability  (IOC)  is  unclear. 

Some  confusion  was  expressed  regarding  the  definition  of  IOC. 
Apparently  IOC  has  been  defined  as  (a)  when  the  first  operational 
piece  of  equipment  becomes  available,  and  also  as  (b)  when  the  first 
unit  to  receive  the  new  equipment  becomes  operational.  TRADOC 
Regulation  600-4  clearly  employs  the  second  definition.  If  the  first 
definition  is  used,  the  training  system  would  have  to  be  developed 
much  more  rapidly  than  if  the  second  is  used. 


25.  Persons  assigned  to  a  TSM's  staff  should  be  familiar  with 
the  general  developmental  cycle  for  new  equipment. 

Some  TSMs  and  their  staffs  are  not  familiar  with  the  development 
cycle  for  equipment.  They  noted  that  this  basic  knowledge  should  be 
required  of  TSMs.  Without  it,  they  will  have  difficulty  understanding 
the  LCSMM  and  coordinating  their  training  system  development  activi¬ 
ties  with  equipment  development  schedules. 
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26.  No  satisfactory  procedures  exist  for  preserving  the  "institutional 

experience"  developed  within  a  TSM  office. 

For  most  officers,  serving  as  a  TSM  or  a  staff  member  of  a  TSM 
office  is  a  one-tour  assignment.  Generally  speaking,  each  TSM  office 
contains  four  officers  and  one  civilian  clerk-typist,  none  of  whom 
have  had  prior  experience  related  to  TSM  activities,  and  few  of  whom 
have  ever  had  experience  related  to  the  development  of  training 
systems.  Experience  developed  during  a  tour  in  a  TSM  office  is  lost 
unless  the  TSM  and  his  staff  serve  overlapping  tours  with  their  repla¬ 
cements.  In  some  instances  this  does  occur.  The  use  of  civilian 
aides  to  staff  TSM  offices  was  discussed,  but  this  probably  would  not 
help  much  since  civilians  also  change  job  positions  quite  often.  The 
use  of  overlapping  tours  seems  to  be  the  best  way  to  preserve 
"institutional"  experience. 

Another  way  might  be  to  develop  workshops  for  TSMs,  an  approach 
that  has  been  tried  with  some  success.  As  a  third  alternative,  data 
and  how-to-do-it  information  could  be  made  available  at  the  local 
level  by  means  of  a  management  information  system.  TRADOC  plans  to 
develop  a  CONUS-wide  computer-supported  management  information  system 
makes  this  a  viable  alternative.  Such  a  program  would  allow  for  imme¬ 
diate  update  of  both  procedural  and  methodological  information  and 
encourage  the  exchange  of  lessons  learned  among  TSMs. 


APPENDIX  B 


SUGGESTED  TOPICS/CONTENT  FOR  TSM  HANDBOOK 


1.  Backward  Planning  Guidance 

This  information  would  identify  the  major  milestones  in  the  LCSMM 
and  for  each  describe  the  documents,  studies,  and  other  deliverables 
which  should  be  prepared  by  those  milestones.  If  possible,  this 
guidance  should  include  information  about  lead  times,  and  about  who 
should  be  tasked  to  develop  specific  deliverables.  The  most  important 
milestones  would  be  the  three  sets  of  ASARC/DSARC  reviews  and  the  IOC. 


2.  Proponent  Planning  Guidance 

This  type  of  planning  information  is  somewhat  similar  to  that 
described  above,  but  would  concentrate  on  identifying  for  Training  and 
Course  Development  Directorates  the  materials  they  should  provide  or 
procure  and  the  dates  by  which  they  should  be  ready  for  use.  Items 
which  proponent  schools  have  to  produce/procure  include  facilities, 
instructors,  students  for  various  testing  periods,  instructors  for  OT 
II,  and  so  on.  It  is  important  that  proponent  schools  be  made  aware 
of  such  requirements  early  enough  so  that  the  requirements  can  be 
incorporated  into  their  budgeting  and  scheduling  cycles. 


3.  Information  about  the  Format  and  Content  Requirements  of  Key  Documents 

All  key  documents  which  must  be  produced  as  part  of  the  LCSMM  are 
described  in  a  regulation.  Most  of  the  regulations  describe  format 
and  content  requirements  in  general  terms.  The  interviewees  expressed 
a  need  for  more  detailed  information  about  such  documents,  especially 
about  the  topics,  questions,  and  issues  that  should  be  addressed.  In 
addition,  it  was  noted  that  a  handbook  containing  exemplary  documents 
to  serve  as  models  would  be  useful.  ' 


4.  Guidance  Regarding  the  Preparation/Review  of  RFPs/Contracts 

Typically,  someone  within  a  TSM  office  or  within  a  school  direc¬ 
torate  will  be  asked  to  review  or  even  to  prepare  certain  portions  of 
an  RPF  or  contract.  In  some  Instances  Mil  Standards  contain  material 
that  can  be  Incorporated  Into  an  RFP/contract,  and  a  few  documents 
illustrate  how  certain  requirements  should  be  stated  in  an  RFP.  These 
and  similar  guidance  material  should  be  examined  to  determine  whether 
additional  material  should  be  prepared  to  cover  other  contractual 


requirements.  For  example,  how  should  the  requirement  to  develop  a 
preliminary  Individual  and  Collective  Training  plan  or  a  provisional 
QQPRI  be  stated  in  a  contract? 


5.  Guidance  Regarding  How  to  Monitor  Contractor  Efforts  and  to  Evaluate 

Contract  Deliverables 

For  certain  contractor-developed  deliverables,  guidance  regarding 
how  to  evaluate  them  does  exist.  For  other  documents  and  products, 
guidance  does  not  exist  in  a  readily  usable  form.  Assuming 
that  in  the  future  contractors  will  be  required  to  develop  most  LCSMM- 
related  training  documents  and  products,  it  seems  appropriate  to  iden¬ 
tify  those  deliverables  for  which  monitoring/evaluating  guidance  does 
not  now  exist,  and  then  to  prepare  such  guidance. 


6.  Information  Describing  TSM  Involvement  with  Operational  Testing 

Some  TSMs  have  had  difficulty  clarifying  their  role  during  OT  II. 
Operational  Testing  is  conducted  by  OTEA,  but  TSMs  are  involved  with 
preparations  for  OT  II,  and  are  responsible  for  assuring  that  certain 
training-related  items  are  available.  They  also  are  responsible  for 
assuring  that  appropriate  instructors  and/or  operators  are  available 
at  OT  II.  These  and  other  TSM  requirements  with  regard  to  OT  II  need 
to  be  clearly  established. 


7.  Guidance  Regarding  the  Role  of  Other  TRADOC  Agencies 

TSMs  are  authorized  to  task  other  TRADOC  agencies,  but  this 
tasking  authority  sometimes  is  not  clearly  stated  in  the  TSM  charter. 
In  addition,  many  TSMs  seem  unclear  about  the  services  that  can  be 
provided  to  them  by  other  TRADOC  agencies,  and  the  appropriate  entry 
points  into  these  agencies.  These  and  related  subjects  could  be 
described  more  specifically  in  a  TSM  handbook. 


8.  Guidance  Regarding  the  Conduct  of  CTEAs 

TSMs  are  responsible  for  assuring  that  CTEAs  are  conducted.  At 
present  it  is  not  clear  whit  the  minimum  requirements  of  a  CTEA  are 
and  how  to  approach  the  problem.  Furthermore,  little  guidance  exists 
regarding  how  to  conduct  a  CTEA.  This  problem  Is  the  subject  of  on¬ 
going  studies.  The  best  approach  at  present  seems  to  be  to  rely  on 
past  efforts  as  examples. 
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9.  Guidance  Regarding  the  Rationalization,  Supportabil ity.  Inter¬ 
operability  (RSI)  Program 

Certain  of  our  new  weapons  systems  are  being  developed  in 
cooperation  with  our  allies.  The  goal  is  to  develop  weapons  systems 
which  can  be  supported,  operated,  and  even  manufactured  by  our  allies. 
Under  this  program  a  TSM  has  to  prepare  a  Memorandum  of  Understanding 
with  representatives  of  our  allies.  Any  changes  made  to  prototype 
equipment  and  critical  items  of  training  material  must  be  agreed  to  by 
all  involved  allies.  Those  TSMs  involved  with  the  RSI  program  noted 
the  need  for  information  about  the  program.  They  especially  expressed 
a  need  for  information  about  what,  if  any,  changes  to  prototype  equip¬ 
ment  and  training  devices/material  can  be  made  without  consultation 
with  our  allies.  Though  these  requirements  are  on  a  case  by  case 
basis,  guidance  is  needed  on  the  design,  economic,  and  political  con¬ 
siderations  of  such  changes. 


10.  Guidance  Regarding  How  to  Assure  Consideration  of  Human  Factors 

during  Early  Phases  of  the  LCSMM 

Efforts  to  condense  the  LCSMM  cycle,  and  to  develop  most  training 
related  material  under  contract  may  have  the  effect  of  reducing  human 
factors  considerations  during  the  first  two  phases  of  the  LCSMM.  At 
the  very  least  it  would  seem  that  those  preparing  RFPs/contracts 
should  be  made  aware  of  the  need  to  examine  early  equipment  concepts 
and  prototypes  in  terms  of  potential  human  factors  problems.  The 
RFP/contract  should  emphasize  that  the  contractor  is  responsible  for 
assuring  that  major  human  factors  problems  do  not  exist  in  prototype 
equipment.  Furthermore,  the  RFP/Contract  should  provide  for  the  exa¬ 
mination  of  equipment  concepts  and  prototypes  by  HEL  or  some  other 
suitable  agency.  Information  on  this  subject  could  be  incorporated 
into  a  TSM  handbook. 
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